The ability to wander in a realistic 3D scene is attractive to a wide variety of people, ranging from scientists to kids. A significant problem to allowing such wandering is easy construction and real time rendering of complex 3D scene data. Traditional approaches often involve constructing or recovering a complete geometric and photometric model of the 3D scene by using modeling or vision techniques. Then, simulating the flow of light from a source, the light is reflected from the described model into a simulated camera and onto a film plane to produce an image. Such approaches are attractive as they allow various graphic elements, such as new objects, lights and shadows, to be readily added to the rendered scene. However, typically these approaches are also complex and computationally intensive. For most of the complex scenes, real time systems based on this approach are typically not able to run on high performance graphics workstations, much less typical desktop or portable computers.
Another approach that is becoming increasingly popular is referred to as image based rendering (IBR). In IBR, the 3D scene is represented by a plenoptic function, such as is described by E. H. Adelson and J. Bergen in “The plenoptic function and the elements of early vision”, Computational Models of Visual Processing, pp. 3–20, MIT Press, Cambridge, Mass., 1991. The plenoptic function is constructed discretely by integrating images captured at a set of viewpoints and viewing directions. While rendering, novel views are generated by re-sampling the pre-constructed plenoptic function at given points, along certain paths. Unfortunately, the 7D plenoptic function described by Adelson and Bergen is typically too difficult to construct and store in practice.
More recently, Heung-Yeung Shum and Li-Wei He in “Rendering with concentric mosaics,” Computer Graphics Proceedings, Annual Conference Series, pp. 299–306, Proc. SIGGRAPH'99, August 1999, describe an image rendering process based on concentric mosaics. This image rendering process allows an observer to move freely within a circular region and allows images from the observer's viewpoint to be rendered in a more computationally efficient manner than many prior approaches. However, due to the manner in which images are rendered in the described technique, the quality of rendered images can sometimes suffer. Thus, it would be beneficial to provide a way to improve the quality of such rendered images.
Another aspect of image rendering that has been approached is the rendering of stereo images. In S. Peleg and M. Ben-Ezra, “Stereo panorama with a single camera,” Proc. Computer Vision and Pattern Recognition Conference, 1999, stereo images are generated from a single rotating camera (off-centered). However, in the Peleg and Ben-Ezra system, the user is constrained to a fixed viewpoint—the scene cannot be wandered through by the observer. Thus, it would be beneficial to provide a way to allow stereo rendering of images while wandering through a scene.
The improved image rendering based on concentric mosaics described below addresses these and other disadvantages.